Spatial Pattern Dynamics due to the Fitness Gradient Flux in Evolutionary Games 
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We introduce a non-diffusive spatial coupling term into the replicator equation of evolutionary 
game theory. The spatial flux is based on motion due to local gradients in the relative fitness of 
each strategy, providing a game-dependent alternative to diffusive coupling. We study numerically 
the development of patterns in ID for two-strategy games including the coordination game and the 
prisoner's dilemma, and in 2D for the rock-paper-scissors game. In ID we observe modified travelling 
wave solutions in the presence of diffusion, and asymptotic attracting states under a frozen strategy 
assumption without diffusion. In 2D we observe spiral formation and breakup in the frozen strategy 
rock-paper-scissors game without diffusion. A change of variables appropriate to repHcator dynamics 
is shown to correctly capture the ID asymptotic steady state via a nonlinear diffusion equation. 
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Evolutionary games provide a convenient and promis- 
ing method for modeling the dynamics of selection and 
competition in biological and social processes pQ [5]. 
While game theory at its origin focused on individual 
strategic decisions in a game played once 0, time was 
later introduced in terms of members of a population who 
play repeatedly, and change their strategy according to 
some rule [1] , or equivalently in terms of changes in the 
frequency of inherited traits or the genes that represent 
them fJj. This dynamic approach has already shed light 
on how group interactions can support traits that are not 
obviously advantageous [SJ E] . 

One way to mathematically model the evolution of fre- 
quencies of a given strategy within a population is via the 
replicator equation [3 [5]. For strategies i — 1, ...,m, the 
replicator dynamics for the ith strategy are defined by 



dui 
~dt 



/.(u) - /(u) 



(1) 



where u{t) = {ui, ...,Um) denotes the frequencies (num- 
ber of players) of each strategy in the population with 
^ = 1. The fitness of strategy i is written /i(u), and 
the average fitness of the entire population is /(u). In 
evolutionary games, the fitness of a strategy depends not 
only on the payoff for that strategy, but also on the fre- 
quencies of every other strategy in the population. If A 
is the mx m payoff matrix for a symmetric game, then a 
natural choice for the fitness function is fi = (^u)^, the 
expected payoff for an individual playing strategy i, and 
/ = u^Au is the scalar average payoff for all strategies. 

Diffusive spatial coupling or random mobility in evolu- 
tionary games allows for pattern formation and gives rise 
to interesting and rich dynamics; it can also support out- 
comes not attained without the inclusion of spatial effects 
[QHTT]. For example, an evolutionarily stable strategy 



(ESS) is one which cannot be invaded by another strat- 
egy at an initially small frequency or population level ^ . 
However, it has been shown in a spatial setting that an 
ESS can invade and replace another ESS via a diffusion- 
induced traveling wavefront [12]. In public goods games, 
unequal diffusion of strategies can promote cooperative 
behavior by allowing the productive cooperators to ag- 
gregate and coexist among the more rapidly diffusing 
freeloaders, similar to a Turing instability [131 [T3]. Other 
approaches to introducing spatial dimensions into evolu- 
tionary games include fixed individuals on a grid playing 
against nearest neighbors [TBI HI] ) or against randomly- 
chosen neighbors while allowing for position swaps |20|. 
The inclusion of diffusive coupling in the replicator ODEs 
corresponds to an assumption that all individuals in the 
population wander randomly while changing strategies. 
This means that the local flux of those playing the zth 
strategy is given by Fick's Law: Ji = —DiVui, with 
Di the diffusion constant. What results is essentially 
a reaction-diffusion system, where the "reaction" comes 
from the competitive interaction between players [9l llOj . 



Once space is included, the frequency interpretation 
of u no longer holds, because ^ Ui varies from point to 
point; we can still treat u(a;, t) as the vector of local popu- 
lation densities, and ||u|| i = ^ the total density (num- 
ber of players) at {x,t). Following the approach taken 
in diffusive evolutionary games by Vickers [S], we define 
the fitness /i(u) — {A\i)i/\\\i\\i, the expected payoff for 
strategy «, and the average payoff /(u) = u'^Au/||u||f . 

What if the players do not move randomly? A more 
general transport law for spatial evolutionary games is 
given by the following expression for the fiux 



J, = -AVu, + l3,u,V [(ylu)i/||u|| 



(2) 
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Here f3i is the proportionality constant for the fitness gra- 
dient V/i, which allows for the spatial movement of play- 
ers in the direction of increasing payoff. Including a flux 
of players moving in a profitable direction, as an alter- 
native to changing strategies, is a sort of spatial version 
of the replicator equation time dynamics. The same ap- 
proach has been taken in modeling dispersal and spatial 
distribution in ecology [HJ [52]; a related idea was in- 
troduced as "success-driven migration" in a prisoner's 
dilemma model on a grid [23], which has recently been 
extended to model reputation-based migration [21] . Here 
we use the flux in Eq. [2] to obtain the partial differential 
equation (PDE) system: 



duj 
dt 



- /3,V • KV[(Au),/||u||i]) + AAii„ (3) 



where at is a proportionality constant for the game. The 
derivation of the fitness gradient flux begins with the 
assumption that the probability of a player moving is 
proportional to the local difference between the fitness 
at adjacent points in space |25j . In the continuum limit 
this contributes a term V/i, which appears in the flux 
multiplied by the local player density u^, proportional to 
the number of players with fitness fi. 

Here we present a study of the consequences of this 
non-diffusive spatial coupling in several 2x2 and 3x3 
symmetric games. Throughout this paper we will use 
/3i = /32 = /3, in other words we will not study the pos- 
sibility of different flux coefficients of response to the 
fitness gradient. Similarly we will take ai = a2 — a 
for the replicator term coefficients. We will however 
allow for differences in diffusion, Di ^ D2- For con- 
venience, we will use the following notation for 2x2 
games: ui = u,U2 = v,Di = D^, etc. Our numerical 
results were obtained with a fully-implicit scheme using 
a parallelized Newton iteration, on spatial grids of 256 to 
4048 points (ID) or 256 x 256 points (2D). Semi-implicit 
schemes are frequently employed in reaction diffusion sys- 
tems, allowing the nonlinear reaction to be handled ex- 
plicitly |15j . Since the spatial derivatives in our system 
are nonlinear, an implicit treatment of the nonlincarity 
cannot be avoided. 

To study the effects of the fitness gradient flux on pre- 
viously known results, we compare with phenomena ob- 
tained by Hutson and Vickers in the setting of a pure 
coordination game in one dimension [12'; we find that 
the fitness gradient flux modifies their results strikingly. 
In a pure coordination game, strategies prefer their own 

an 



type, as is evident from its payoff matrix A 



022 



If a22 > flu, it would be more advantageous for the pop- 
ulation to play the 2nd {v) strategy, nonetheless a pop- 
ulation uniformly playing the u strategy is known to be 
stable (ESS), and cannot be invaded by a small popula- 
tion playing the other strategy. In fact each pure strat- 
egy is evolutionarily stable i27j. However, Hutson and 
Vickers proved that diffusive coupling can produce trav- 
elling wavefronts, whereby a population v invades u when 
0-22 > Oil [13 ■ The initial conditions they considered 
were two adjacent regions of pure strategy dominance 
with no overlap (our initial conditions will be smooth 
functions with a small overlap, see e.g. Fig. [T^). In this 
adjacent region geometry, diffusion initiates replicator 
dynamics as the sharp initial population distributions re- 
lax and overlap with time. Depending on the diffusion 
constants, a local increase or bump in u (Dy > D^) or v 
[Du > Dy) occurs at the region boundary in the travel- 
ling wave solution [T^ . 

The fitness gradient flux adds spatial gradient terms 
which may modify existence or other properties of these 
solutions. From a qualitative standpoint, the diffusive 
flux between the adjacent region can now be enhanced 
or opposed by the fitness gradient flux. Generally, the 
PDEs in Eq. [3] can be shown to be normally parabolic 
[25] . and thus well-posed for (3 values such that 



0<(3< 



Dy{aii - 012) + Du{a22 - 021) 



/3c; (4) 



in other words, for strong enough diffusion. The equation 
has smooth solutions in this range of but solutions 
blow up as f3 approaches /3c [111 E] . 

We numerically simulate Eq. [3]for a coordination game 
with Du = f, Dy = 3, a = I, and payoff matrix A = 

Q 2 ' ill which case f3c — 2.4. We study Hutson- Vickers 

solutions in an adjacent region geometry for /3 < /3c, and 
find that the travelling wave continues to exist for /3 > 0, 
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FIG. 1. Spatial dynamics of a travelling wave in the adjacent 
region geometry for a coordination game with payoff matrix 
given in text, a — 1, Du ~ 1, -Du ~ 3, and /? — 0.2. Times 
shown are as labelled. 
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FIG. 2. Effects of tlie fitness gradient flux on fully developed 
travelling wave solutions in the coordination game: a) u{x, t), 
b) v{x,t) at t — 390.2, for initial conditions in the adjacent 
region geometry as in Fig. [l] Here Du ~ 1, Dv — 3, and 
a = 1; values of as labelled. Note that for both /? = 1 and 
2, the solutions have reached a steady state (speed c — 0). 



albeit with the observed speed of translation c reduced. 
An example of the development of such a travelling wave 
solution is shown in Fig[T] for /3 = 0.2. Note that a bump 
is still observed in u, corresponding to the condition Dy = 
3 > Du [H]. As /? is increased, the wave speed c of these 
solutions appears to decreases linearly with P (for (3 = 
0.1 — 0.6), and stops entirely for /? > 0.7, see Fig [2] For 
those cases with c — 0, the wavefront develops initially, 
but the invasion is halted and the solution achieves a 
steady state (/3 = 1 and 2 in Fig [5]). 

Moreover, we observe similar travelling wave solutions 
numerically in the prisoner's dilemma game, which was 
not considered in I12j . An example is shown in Fig. |3j 
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FIG. 3. Spatial dynamics of a travelling wave in the adjacent 
region geometry for a prisoner's dilemma game with payoff 
matrix given in text, a = 1, Du = -Du = 0.1, and P = 1. 
Times shown are as labelled. 
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FIG. 4. Top row: ID spatial dynamics of a random mode per- 
turbation of u and v for a prisoner's dilemma game with no re- 
action or diffusion, showing the progress towards a structured 
steady state, times as labelled. Bottom row: corresponding 
evolution of the proportionate variable u[x, t). Here hi[A) = 1 
(see text). 
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and P — \. This is quite different from 



the purely diffusive coupling case (/3 = 0), for which no 
travelling waves are seen. Instead, the system progresses 
towards the only Nash equilibrium of the game (everyone 
defects, which is also an ESS) [27^, with dynamics essen- 
tially unmodified by the spatial coupling. The travelling 
wave which occurs with the inclusion of fitness gradient 
flux may be caused by the directed motion of coopera- 
tors, who can now flee in front of the takeover of defec- 
tors. We do not yet know if these observations for the 
prisoner's dilemma game will correspond analytically to 
the travelling wave solutions in the coordination game 

We next consider the dynamics of 2-strategy games in 
one-dimension and study the fltness gradient flux in the 
no reaction, no diffusion case (a^ = and Di = hi 
Eq. [3|. This corresponds to a situation in which players 
who cannot change strategy, but may nonetheless move in 
order to improve their benefit — their strategy is "frozen" . 
The dynamics still depend on the properties of the payoff 
matrix A through the fitness gradient flux in Eq. [3j which 
provides a game-dependent spatial coupling. 

We find numerically that for certain classes of games, 
characterized by a simple condition on the matrix A dis- 
cussed below, the frozen strategy assumption leads to a 
spatially structured steady state. An example is shown 
in Fig. [4] for random initial conditions in the prisoner's 
[1 -ll 



dilemma game, A — 



3 



Note that this steady state 



does not represent the coexistence of opposing strategies 
in the usual sense (cooperation), because by assumption 
players are not allowed to change strategy. 

To begin to understand this steady state, we let ui = 



4 



u, U2 = V, and define the proportionate variable 
u(x,t) u{x,t) 



u{x, t) 



u{x, t) + v{x, t) ' 



with a similar definition for v{x,t). Because u + w = 
1 at each point, only one of these variables is needed. 
However, we do not obtain a new PDE purely in terms 
of u. Defining p{x, t) — \Iu/{u + w), we obtain 

dfU = L{u, p, A) ■Vu + K{A)bu{l- u)Au, (5) 

where L is a nonlinear function, > is a constant, and 
for payoff matrix elements a^j, the constant 

k{A) EE ai2 + a2i - (an + 022) 

controls the spatial dynamics [55]. For k{A) < 0, Eq. [S] 
is ill-posed as a backwards heat equation, which corre- 
sponds to cooperators aggregating faster than defectors 
can follow as the solutions blows up in finite time. For 
cases with k{A) > 0, we find that solutions evolve to a 
steady state defined by Vu — 0, which defines a steady 
state solution to Eq. [5]- an example is shown in Fig. |4] 

It is interesting to consider solutions to Eq. [5] in ref- 
erence to the classification of two-strategy matrix games 
proposed by Szabo & Fath (Sect. 2.4 in [ST]): 

1. Anti-coordination class: games with an < 021 and 
022 < ai2. The coefficient k{A) > for all games in 
this class and the system evolves to a steady state 
as each population locally adjusts to the relative 
benefit between its own type and the other. 

2. Coordination class: games with an > 021 and 
022 > ai2, so k{A) < 0. Under the fitness gra- 
dient fiux, populations will seek to aggregate by 
strategy type. Well-posedness in this case can only 
occur in the presence of diffusion, with sufficiently 
high coefficients equivalent to the /3 < /3c condition 
discussed above. 

3. Pure-dominance class: games with (an — a2i)(a22 — 
112) < 0. The coefficient k{A) can be of either 
sign here, so well-posedness depends on the specific 
choice of A, sufficiently high diffusion, or other as- 
sumptions. In the prisoner's dilemma, cooperators 
u aggregate locally, while defectors v follow them. 
If A is chosen so that k{A) > 0, as in Fig. |4j then 
the system evolves toward a steady state - the local 
aggregation of u into four peaks is readily apparent 
in that case. For games with k{A) < 0, the PDE is 
ill-posed. 

Three strategy games allow for the possibility of cyclic 
dominance of strategies [28] . where any given strategy 



can be beaten by another. We investigate the dynamics 
of the three strategy rock-paper-scissors game [2S] un- 
der the frozen strategy assumption, defined by the payoff 
matrix 



A = 



-1 1 ■ 

1 -1 
-1 1 



(6) 



We observe spiral waves in this system, as shown in 
Fig. [5] with periodic boundary conditions; spiral waves 
in the rock-paper-scissors games are reported elsewhere 
for other spatial couplings [20] [29] . Because it is difficult 
to represent all three strategy populations on a single 
plot, we use the following normalized version of the ad- 
ditive RGB color system. We first assign a color to each 
strategy; at each (x,i), the u,v,w values are normalized 
such that the minimum is 0, the maximum is 1, and the 
other is in the closed interval [0, 1]. These numbers de- 
termine the RGB value for that point [30]. The result is 
an image indicating which strategy is predominant, and 
the relative proportion of the other two (an example of 
a single strategy concentration field for a spiral wave is 
shown in Fig. |6ji). 

The initial conditions consist of a superposition of 
Gaussian bumps for each strategy population, of the form 



(x, 0) = Qe 



-Ki{x- 



1,2,3 (7) 




FIG. 5. Spiral pattern in the frozen-strategy rock-paper- 
scissors game, showing its full extent a,t t — 5.0; strategy 
fields u, V, w are shown in an additive RGB color map (see 
text). The inset shows a closeup of the initial conditions. 
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FIG. 6. Development of a spiral pattern, for times a.) t — 0.5, 
b) 1.6, c) 3.0, with strategy fields shown in an additive RGB 
color map; d) corresponding heat map plot for one individual 
strategy ui{x,t) is shown, also at t = 3.0. 



To produce a spiral pattern, the points X; are placed 
symmetrically on a circle of radius r about the origin [26j . 
In Fig. [5) r = 2 with K, ^ 5 and d = 1 for all i (256 
X 256, periodic boundary conditions. At = 0.01). Spiral 
formation is determined by the proximity and magnitude 
of these bumps, via the parameters r and Ci. For fixed 
Ci, there is a range ri < r < r2 where spiral formation 
is orderly and persists until boundary effects cause it to 
break up; this breakup is already evident at the edges 
of Fig. [5] An example of spiral development is shown 
in Fig. |6^-c. The pattern rotates as the three Gaussian 
bumps chase each other, driven by the fitness gradient 
flux, and density waves are radiated forming the spiral 
arms. When the initial bumps are close (small r), the 
spiral forms right away, but breaks up before it reaches 
the boundaries. If the bumps are further apart (large r), 
formation proceeds more slowly, and is incomplete before 
the pattern is complicated by boundary effects pS] . 

To test the robustness of the spiral to asymmetric ini- 
tial conditions, we varied the size Ki of the Gaussian 
bump for each strategy, and the payoffs by replacing the 
±1 in Eq. [6] with values drawn from a uniform random 
distribution on [0,1]. While these changes introduced 
some asymmetries, they did not stop spiral formation 
from the three bump configuration. 

Spiral waves are observed as a generic feature in 
many reaction-diffusion systems such as the Belousov- 



Zhabotinsky reaction [3TJ[35], the Fithugh-Nagumo equa- 
tions |33) , as well as in the amoeba system Dictyostelium 
|34| . Here a rotating spiral results from specific initial 
conditions, the cyclic nature of the rock-paper-scissors 
game, and the flux in Eq. [2] which drives players to- 
wards gain (e.g. scissors chases paper) and away from 
loss (e.g. scissors flees from rock). We also note that 
realizations of this game in nature, for instance the mat- 
ing strategies of a California iguanid lizard |35) . may be 
well described by our mathematical hypothesis, as such 
creatures cannot change their strategy. 

The spiral is however not a completely robust solution 
in our system - it is not an attractor for all initial condi- 
tions. Fig. [7] shows a simulation at t ~ 5.0 of the frozen- 
strategy rock-paper-scissors game, with random initial 
condition: the superposition of Gaussian bumps in the 
form of Eq. [7] centered oir randomly distributed points 
Xj. We do not observe a spontaneous organization into 
spiral waves, as is often seen in other systems Since 
for all finite domains we have tested, a single spiral wave 
eventually breaks up. Fig. [7] may be more characteris- 
tic of the asymptotic state for this spatial game. The 
complex pattern of predominant strategy distribution in 
this rock-paper-scissors game does bear a striking resem- 
blance to the orientation map of visual selectivity in the 
primary visual cortex of the brain |36j . 

In this paper we have studied the effects of a fitness gra- 
dient flux on spatial PDEs for evolutionary games with 
replicator dynamics, comparing the effects of this new 
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FIG. 7. Disordered state in the rock-paper-scissors game, at 
t = 5.0, shown in additive RGB color. 
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term to diffusive coupling. We have seen numerically 
that the fitness gradient flux can slow or stop travelling 
waves in some cases, and lead to new travelling wave 
solutions in others. We have also introduced a PDE sys- 
tem with no diffusion and no strategy-changing dynam- 
ics, and studied some of the patterns that result in this 
frozen strategy case. The progression of initial conditions 
towards some final pattern characterized by a constant 
value of the proportionate populations Ui/^u remains 
to be understood in terms of stability and basins of at- 
traction. For frozen strategy dynamics in three strategy 
games (here rock-paper-scissors), the mathematical con- 
ditions for spiral stability and instability also remain to 
be understood. 

In considering the relative motion of cells in growing 
embryos or tumors, a number of different aspects have 
been included in mathematical models of spatial motion, 
such as diffusion, chemotaxis, and differential adhesion 
[571 155] . while game theory has been applied to some 
competitive interactions |39j . In this context, the equa- 
tions proposed here provide a way to combine motion 
and competition among players (cells). Such a directed 
transport may be more appropriate to spatial game the- 
ory models of biological or social systems than a random 
(diffusive) flux. 

We would like to thank R. H. Austin for early inspira- 
tion leading to this work, and J. P. Keener, D. H. Kelley, 
T. Reluga, and M. J. Shelley for helpful comments and 
discussion. 
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